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APPARATUS AJ\D METHOD FOR FUSION AND IN-OPERATING- 
ROOM PRESENTATION OF VOLUMETRIC DATA AND 3-D 

ANGIOGRAPHIC DATA 
BACKGROUND OF THE INVENTION 
5 FIELD OF THE INVENTION 

The present invention relates to medical imaging systems in general, 
and to an apparatus and method for presenting in-room, real-time updated 3- 
dimensional arteries model, including plaque. 

10 DISCUSSION OF THE RELATED ART 

Medical imaging devices are widely used for a number of purposes, 
both prior to and during medical operations, Pre-Operation purposes include 
evaluation of a patient's status, assessment of required treatment, treatment 
planning in general and catheterization in particular. During-operation purposes 

15 include on-going assessment of the patient's condition, and locating the exact 
position of invasive tools and devices. 

Any currently existing imaging modality has its strengths and 
weaknesses. Angiogxams have high resolution, which enables to depict small 
vessels (with diameter smaller than 0.8mm), mmoticeable in other modalities. The 

20 high resolution of angiograms and three-dimensional products of angiograms 
processing, provide accurate measurements of distances, such as arteries 
diameter. Angiogramis are also up-to-date in nature. However, soft tissues are not 
visible in angiograms so a lot of information, and plaque information in particular 
is missing when coming to describe in details the current state of arteries. 

25 CT scanners, on the other hand, provide volimietric information and 

therefore offer 3-dimensional presentation of segmented infonnation, including 
soft tissues in general and plaque deposited along the arteries in particular. 
However, CT scans lack up-to-date information, since they are taken prior to an 
operation. In addition, the resolution of CTs is mferior to the resolution provided 

30 by angiograms. 
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CT scans enable the reconstruction of the vessels structure by tracking 
the lumen, i.e., the space inside the arteries. The shortcoming of this constructioix 
is that if a blood vessel is blocked, little or no blood flows through it and th& 
relevant part of the blood vessel can not be visually reached by tracking th& 
5 lumen. In addition, the blood vessels' walls can be viewed in CT scans provided- 
they are at least 1.5 mm wide (the width of a healthy coronary artery, for 
example, is 100-900|i). Therefore, it might be impossible to tell that a blood, 
vessel carries significant sediments until it is substantially damaged, or to assess 
the percentage of idie stenosis of the blood vessel. Using the standard tools, it is 

10 only possible to tell if stenosis takes up more or less than 50% of the vessel's 
diameter. If the vessel wall is 2mm or greater, a more precise estimation of th& 
percentage of the stenosis can be provided. 

There is therefore a need in the art for a system that will combine the 
best of CT with the best of angiograms to supply in-operating-room, up-to-date, 

15 accurate 3-dimensional information. 
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SUMMARY OF THE PRESENT INVENTION 
In accordance with a first aspect of the present invention, there is 
provided an apparatus for displaying a first image, said first image is a product 
of processing a second image taken by a medical imaging device prior to a 
5 medical operation, the first image comprising information about areas having 
sediments, wherein during the operation, said first image is presented to a user 
of the system. The first image is fused with a third image said third image is a 
product of processing a fourth image taken by a medical imaging device during 
the operation. The apparatus further comprises a computer program for 

10 registration of images of the medical imaging devices; and fusing information 
contained in and associated with products of processing images of the medical 
imaging devices; and presenting the first or third or a combination of the first 
and third images containing information obtained fi-om the medical imaging 
devices or an image containing information obtained fi*om the either one of the 

15 medical imaging devices. The apparatus also comprising a correction module 
to correct imaging errors in the second image, using the fourth image acquired 
during the operation. The imagiag error is characterized by one or more 
calcified areas of one or more blood vessels depicted outsized on an image, 
said image is a product of processing of one or more images taken by one or 

20 more medical imaging device prior to an operation. In the preferred 
embodiment the first image and the third image, are presented on the same 
location on the visual display, where tiie first and the third images are partially 
transparent At least a part of the first image, and at least a part of the third 
image can be presented adjacent to each other. Sediments foimd by processing 

25 the first image are distinctively marked on the third image. The sediments 
found by processing the first image are distinctively marked on the first image. 
The apparatus further comprising a marking module for marking one non- 
flexible part of the blood vessel, on the first or third image. The apparatus 
further comprising a module for marking the at least one curved part of the 

30 blood vessel, on the first or third image. The apparatus further comprises a 
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module for marking indications prepared prior to the operation, on the first or 
third image. The apparatus further comprises a module for indicating during an 
operation, parameters determined prior to the operation, for a medical imaging 
device, said parameters to be applied while taking images. The apparatus 
further comprises a module fon identifying one point in an image presented 
during the operation with one or more check-points indicated prior to the 
operation; and presenting the image, associated prior to the operation with the 
one or more check-points. The blood vessel can be a coronary artery. The 
sediments can be any one of the following: lipid-rich plaque, intermediate 
plaque, calcified plaque, thrombi, cells or products of cells. The medical 
imaging device can be a multi slice computerized tomography device. 

In accordance with a second aspect of the present invention there is 
provided an apparatus for detecting a part of a blood vessel with sedim^its, 
from a first image acquired by an imaging device prior to an operation, the 
apparatus comprises: an identification module for identifying the part of the 
blood vessel, and within said part the sediments located fherem; and a marking 
module for indicating the part of the blood vessel and sediment associated 
therewith on a second image created by processing images taken by a medical 
imaging device. The identification module is receiving intensity values for tiie 
pixel of the image acquired by a medical imagmg device; and range of 
intensity values for the type of sediment. The apparatus further comprises a 
module for constructing a visual representation of the lumen of the blood 
vessel. The apparatus fiarther con:Q)rising a module for constructing a visual 
representation of the part of the wall of the blood vessel. The parts of the blood 
vessel and the sediment submerged tiierein are indicated using color-coding. 
The apparatus further comprising a width determination module for 
determining the width of the sediment layers at a location along a blood vessel; 
and the diameter of a blood vessel at a location along the blood vessel; and the 
percentage of stenosis of a blood vessel at a location along the blood vessel. 
The widths of the sediment layers, the diameter of the blood vessel and the 
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percentage of stenosis are indicated on the second image. The apparatus further 
comprises a module for indicating, in response to a user action, a part of the 
blood vessel as non-flexible. The apparatus further enabling comprises a 
module for indicating, in response to a user action, a part of the blood vessel as 
5 curved. The apparatus further comprises a check-point definition module for 
indicating, in response to a user action, a position within the body of a patient 
as a check-point and associate said check-pomt with the image produced by 
processing the first image. The second image depicts a three-dimensional view 
of a part of the blood vessel. The second image depicts a surface within the 

10 human body and the blood vessel on said surface. The second image depicts an 
internal three-dimensional view of the blood vessel. The second image depicts 
a cross-section of a blood vessel at a location along the blood vessel, said 
cross-section comprising one or more of the followmg: the blood vessel's wall, 
the lumen of the blood vessel, sediments submCTged on the blood vessel's wall. 

15 The apparatus further comprises a module for manually correcting the 
indications for sediments on images acquired prior to an operation and the 
products of said images. The correction includes changiag the size or the 
sediment type of an indication, adding, or deleting indications. 

In accordance with a third aspect of the present invention there is 

20 provided a method for displaying a first image, said first image is a product of 
processing a second image taken by a medical imaging device prior to a 
medical operation, the image comprising information about areas with 
sediments, during the operation. The first image is fused with a third image 
which is a product of processing a fourth image taken by a second medical 

25 imaging device during the operation, the method comprises the following 
steps: registering the coordinate systems of the first and second medical 
imaging devices; and fusing information contained in and associated with 
products of processing images of the first or second medical imaging devices; 
and presenting an image from the first or second medical imaging devices or an 

30 image containing information of images firom the first and second medical 
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imaging devices. The registration of the coordinate systems comprises the 
following steps: matching three or more points seen in images of tihie first and 
second medical imaging devices; and matching the coordinate frames of the 
first and second medical imaging devices. The matching of the points is based 
on comparing the coordinates of three or more non-aligned fiducials as seen in 
the image of each of Hie two medical imaging devices. The matching of the 
three points is based on comparing an at least one two-dimensional image 
taken during an operation to an at least one projection of a three dimensional 
image constructed firom at least two two-dimensional images taken by an at 
least one medical imaging device prior to the operation. The method fiuthar 
comprising a step of correcting an imaging error of the image taken by a 
medical imaging device prior to an operation, using the image acquired during 
the operation. The imaging error is characterized by a calcified area of a blood 
vessel depicted outsized on one or more products of processing of the image 
taken by a medical imaging device prior to an operation. The registration of tiie 
coordinate systems comprises the followmg steps global registration of a first 
image and a second image taken by the first and the second medical hnaging 
devices; and removal of local residual discrepancies by matching 
corresponding features detected in the first and the second images. The global 
registration is based on comparing the coordinates of a fiducial as seen in the 
first and the second imagra. The global registration is based on matching one 
or more two-dimensional images taken during an operation to a projection of a 
three dimensional data obtained firom the medical imaging device prior to the 
operation. The first image and the third image, are presented on the same 
location on the visual display, where the first and the third images are at least 
partially transparent. The method fiorther comprising a step of marking non- 
flexible part of a blood vessel, on a first image. The method fiirther comprising 
a step of marking non-flexible part of a blood vessel, on the first rniage. The 
method fiirther comprising a step of marldng curved portion of the blood 
vessel, on firet image. The method fiirther comprising a step of marking curved 
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portion of the blood vessel, on the third image. The method further comprising 
the steps of identifying a point in an image acquired during the operation with a 
check-point indicated prior to the operation; and presenting the image 
associated prior to the operation with the check-point. 
5 In accordance with a fourth aspect of the present invention there is 

provided a method for automatic reconstruction of a three-dimensional objects 
from two angiograms using CT information, the method comprises the 
following steps taking a first and a second angiograms of the required area 
from different perspectives; and for the first and the second angiogranfi, 

10 obtaining a first and a second projected CT images by projecting the three 
dimensional CT data on the same plane as the first and the second angiogram; 
and registration of the first and the second angiogram with the corresponding 
projected CT images by objects appearing in the angiogram and in the 
projected CT; and mutual co-registration of the first and the second 

15 angiograms; and detecting objects appearing in the angiogram and match with 
the corresponding objects in the projected CT; and deriving the three 
dhnensional coordinates of the objects appearing in the first and the second 
angiograms; and constmcting a three dimensional image of the required area 
from the first and the second angiogram, 

20 
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BRIEF DESCRIPTION OF THE DRAWINGS 
The present invention will be understood and appreciated more fully 
from the following detailed description taken in conjunction with the drawings in 
whicli: 

Fig. 1 is a schematic block diagram of the proposed apparatus, in 
accordance with the preferred embodiment of the present invention; 

Fig, 2 is a schematic block diagram of the operating components of the 
pre-operation modules, in accordance with the preferred embodiment of the 
present invention; 

Fig. 3 is a schematic block diagram of the image fusion method; and 

Fig. 4 is a schematic block diagram of the operating components of the 
durixig-operation modules, in accordance with the preferred embodiment of the 
present invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

An apparatus and method for fusing images and information about 
tubular organs, from CT scans and angiograms, and presentation of the same in 3- 
dimensions during medical operations is disclosed. The presented information 
5 includes different types of sediments deposited inside and outside coronary 
arteries or other blood vessels, as part of the whole structure of Ihe blood vessels. 
The apparatus is designed to be used both before and during a medical operation, 
usually a catheterization, and also to enable the user to mark different areas of 
interest and pre-defined views prior to the operation. The areas and views will be 
10 presented by the system dining an operation. 

The preferred embodiment of this invention uses slices taken by a 
Multi-Slice Computerized Tomography (MSCT) device. The MSCT scanner can 
simultaneously acqxiire up to 32, 40, or even 64 slices, thus covering the whole 
heart area by slices 0.6 mm apart, that were takan during a time frame of 10-20 
15 seconds. Therefore, the scanner enables high-resolution morphologic evaluation 
of the myocardixzm and the coronary arteries as well as of other blood vessels. 
The MSCT yields a pixel size of 0.3-0.5mm and temporal resolution of 90- 
120mSec. 

Referring now to Fig. 1 that shows an exemplary environment in 
20 which the proposed apparatus and associated methods are implemented. In the 
present non-limiting example, the environment is a cardiologic department of a 
health care institute. A patient who is suspected (or known) to suffer from a 
coronary arteries problem, or another problem related to sediments on blood 
vessels, goes throu^ a scanning session by a medical imaging device. A possible 
25 conclusion of the physician evaluating the images taken by the device is that the 
patient should be catheterized. In this case, the proposed invention discloses how 
to fuse and present images taken or generated prior to tiie operation, with unages 
taken or generated during the operation. When relating to images taken prior to or 
during an operation, the discussion includes both the images as taken, and 
30 products of processing the taken images. The products are model of different 
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body parts or body tissues, such as a vascvdar tree, a heart muscle, plaque or the 
like. The structures may be described as collections of volume elements, tubular 
organs given by lines and radii, surface and so on. The mentioned structures, of 
course, are associated witih one or more visual presentations. 

In the framework of this exemplary system, the pre-opieration input to 
the system comprises images of a body part, for example the heart area of patient, 
taken by an MSCT scanner (not shown). The original images, as scanned by the 
MSCT (also referred to as slices) are stored on a storage device 20. The additional 
images and information storage 30 stores images and other information produced 
by processing the origirtal images. This processing is initiated by the user's 
actions and is performed by the pre-operation work station 40. The angiogram 
and the 3-dimensional reconstructions of vessels based on the angiograms, are 
used during the operation, by the during-operation work station 50 and are stored 
in the angiogram and 3-dimensional reconstruction storage 80. Each of the pre- 
operation work station 40 and the during-operation work station 50 is. preferably a 
computing platform, suchi as a personal computer, a mainframe computer, or any 
other type of computing platform that is provisioned with a memory device, a 
CPU or microprocessor device (not shown), and several I/O ports (not shown). 
Alternatively, the pre-operation work station 40 and the during-operation work 
station 50 can be DSP chips, ASIC devices storing the commands and data 
necessary to execute the methods of the present invention, or the like. The pre- 
operation work station 40 and the during-operation work-station 50 are further 
equipped witii standard means for coUecting input from the user and presenting 
the results 60 and 70 correspondingly. In tiie exemplary environment of the 
present application, these would usually comprise a keyboar4 a pointing device 
such as a mouse, and a display device. The pre-operation work station 40 and the 
during-operation work-station 50 can further include internal storage devices (not 
shown), storing the computer appUcations associated with the present invention. 
These storage devices can also serve as tiie original images storage 20, the 
additional images and information storage 30 and the angiograms and 3- 
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diemensional reconstruction storage 80. The storage xinits 20, 30 and 80 can be 
magnetic tapes, magnetic discs, optical discs, laser discs, mass-storage devices, or 
the like. The computer application associated with the present invention is a set of 
logically inter-related computer programs and associated data stmctures that 
5 interact to perform the tasks detailed hereinafter. The pre-operation work station 
40 and the during-operation work station 50 can be the same machine, separate 
machines and even different machines. Optionally, the pre-operation work station 
40 and the during-operation work station obtain the original images, or store and 
obtain the images and information firoxn sources other tiban the original image 

10 storage 20, the manipulated images and info storage 30, and the angiograms and 
3-diemnsional reconstruction storage 80, such as a remote source, a remote or 
local network, a satellite, a floppy disc, a removable device and the like. 

Further note should be taken that the apparatus presented is exemplary 
only. In other preferred embodiments of the present invention, the computer 

15 applications, the origmal images storage 20, the manipulated images and 
additional information storage 30, the angiograms and 3-dimensional 
reconstractions storage 80, the pre-operation work station 40 and the dxiring 
operation work station 50 could be co-located on the same computing platform. 
As a result, one of the I/O sets 60 or 70 will be rendered unnecessary. 

20 Referring now to Fig. 2 that shows the various modules that perform 

actions on the MSCT images, prior to an operation. It is important to get a 
good understanding of the features and tools available in the pre-operation stage, 
because the products of these tools are used in the during-operation stage detailed 
below. The pre-operation modules are ciivided into automatic modules 22 that do 

25 not require user interaction diaring their work, and mixed modules 23, in which 
the system executes commands in response to the user's actions and inputs. The 
user is typically a physician or a skilled technician. This division to automatic and 
mixed tools is for clarity reasons pnly^ and does not imply order of activation, 
precedence or the like. The modules to be activated and their order depend on the 

30 user's choice. In addition, software engineering considerations naight cause some 
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functionality of certain modules to be called automatically from other modules. 
The products of all the modules of Fig. 2 are stores in the additional images and 
information storage 30 of Fig. 1. 

The automatic modules 22 comprise a number of inter-related 
5 computer implemented modules. The standard 3-D presentation tools module 220 
can be a computer program for presenting 3-D images, provided by the MSCT 
manufactures and also by independent manufactures such as VITREA® 
manufactured by Vital Images, Plymouth, MN, USA. The data acquired by CT 
scanners is volumetric in nature, i.e. intensity information is associated with each 

10 volume xmit, named voxel. Since each substance scanned has a specific range of 
intensity, the intensity data represents the composition of the area seamed. The 
CT intensity is measured in Hounsfield Units (HU). This raw volumetric 
information enables the reconstruction of segmented information, i.e. 
reconstraction of specific body parts and tissues. Th.e presentation tools 220 

15 enable a number of processing and viewing options of tfcie scanned images. 

One option is to show visible two-dimensional or three-dimensional 
presentations of the models of various body parts and tissues, the models 
constructed from at least two slices scanned by the device. Two-dimensional 
images include, for example, scanned shoes. Three-dimensional images include, 

20 for example, images of surfaces, images of an arteries structure and the like. 
Another presentation option involves producmg ne-w planar images, either 
parallel or at a predetermined angle to shoes taken hy the imaging device. Yet 
another option is presenting a cross section of an artery, or even a sequence of 
such cross sections, thus visualizing "fly through". 

25 One more option is to present one or more images in various layouts, 

such as presenting at least two adjacent images depiciing the same or adjacent 
locations, presenting images in a temporal sequence, and the like. 

The above mentioned views, images, and their combinations are stored 
in the additional images and infomxation storage 30 of Fig. 1 . 
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The lumen construction tool 221 can also be a computer program for 
constructing 3D images showing lumen, such as VITREA® manufactured by 
Vital Images, Plymouth, MN, USA. The tool 221 reconstructs the vessels 
structure by tracking the lumen, i.e., the space inside the arteries. As mentioned 

5 earlier, blocked parts of small diameter vessels are unreachable. 

The plaque identification and classification module 223 identifies the 
various types of plaque that might be deposited in the blood vessels. This is 
enabled by the high spatial and temporal resolution of the MSCT device, 
relatively to single-slice CT scanners (the angiograms, although high-resolution, 

10 do not enable soft-tissues imaging). The detection of the sediment type is 
performed by traversing the data structure representing the lumen and comparing 
the intensity data associated with the area adjacent to the lumen with predefined 
intensity ranges. Thus, the detection of sediment is performed by "tracking" the 
blood vessels through the lumen structure constructed by the lumen construction 

15 module 221 as a road map, and comparing the values of the CT intensity fonnd in 
the vicinity of the blood vessels to known ranges (after ignoring the values 
associated with the heart surface). The following table lists exemplary ranges of 
values for each type of sediment: 



Sediment Type 


Intensity (HU) 


Lipid-rich (soft) plaque 


>=12 and <= 62 


Fibroid (intermediate) plaque 


50-130 


Calcified plaque 


120 <= 



20 



The table was presented by Stephen Schroder, Tiibingen, at the " 
Intemational Task Force for Prevention of Coronary Heart Disease" Symposiiun, 
Scuol, Febraary 23, 2003. 

As can be noticed firom the table, some ranges of intensity can be 
25 interpreted in more than one mode. In such cases considerations of continuity 
with surrounding areas will be applied. 
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The blood vessels' walls construction module 223 retrieves the 
infonnation about the blood vessels' walls that can be deduced from the high- 
resolution CT shoes. When using the MSCT technology, each pixel represents a 
square with a side of 0.4mm in average. Due to rounding problems, at least two 
pixels are required in order to detect an edge in general and the wall of the blood 
vessel in particular. Thus, only blood vessels' walls that are at least 0.8mm wide 
can be recognized accurately. For blood vessels with thinner walls, roundmg 
problems cause substantial errors and inhibit the correct presentation. The 
information about the lumen, sediments, and vessels' walls combined togetha:, 
provides an informative view of the blood vessels, and is stored m the additional 
unages and information storage 30 of Fig. 1. 

The mixed (automatic and manual) modules 23 comprise a number of 
inter-related computer implemented modules. The user interface module 229 
presents the user with all the options he or she can chose from when working witii 
the system. The presentation of these options uses graphic, textual or any other 
means. When choosing a certain option, the system enables the user to make the 
relevant choices, perform the relevant actions and store the results. For example, 
. when users select the "check point definition" option, the system would allow the 
user to define a check-point and associate views therewith as is explained below 
in the description of the check-pomt definition and view preparation module 233. 

The parameter setup module 230 is used for setting system parameters 
and user preferences, such as color selection, preferred layouts of images, 
numerical parameters and the like. Such parameters are used by both the 
automatic and tiie mixed tools, both prior to and during the operation. The 
parameters and settings are stored in the additional images and mformation 

storage 30 of Fig. 1. 

The plaque width calculation module 23 1 enables the user to pomt at a 
specific location along a blood vessel, and have the system calculate the actiial 
width of the plaque layers deposited at the location; the actual width of the lumen 
at that location; and the percentage of stenosis, if any, at that location. The 
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Stenosis percentage is determined by 1 minus the ratio between the actual area of 
the cross section of the artery at the required location and the average area of the 
cross section along the artery. This average is determined from the graph 
representing the cross-section's area distally and proximally from the required 
location. All mentioned information - the plaque width, the blood vessel's width 
and the percentage of stenosis are stored in the additional images and information 
storage 30 of Fig. 1. 

The plaque correction module 232 enables the user to manually change 
the type, size, density and shape of any plaque sediment recognized by the 
system. It also enables the user to add or remove indications for plaque. In 
particular, correction might be needed in areas suffering from the blooming 
effect, due to which heavily calcified areas appear outsized. This is caused by the 
reflection of x-rays from the calcified areas onto their neighboring areas. This 
effect is automatically corrected in the during-operation system in the blooming 
effect correction module 254 of Fig. 3. This automatic correction can not be 
performed without additional images of the area, such as angiograms and 3- 
dimensional reconstructions from angiograms, which are usually avaUable only 
during the operation. The user input is accepted through the use of the keyboard 
and the pointing device 60 of Fig. 1. The corrections to the plaque areas are stored 
in the additional images and information storage 30 of Fig. 1 . 

In the check-point definition and view preparation module 233 the user 
can designate any point in the heart area of the patient as a check-point. Since all 
acquired data carries volumetric mformation, each pixel seen in an original slice 
or on certain types of derived images can be uniquely identified with the 
corresponding location in the imaged volume. Clicking with the mouse or 
otherwise pointing at such point defines it as a check-point when the system is at 
the check- point selection mode. The user can then associate one or more views 
with each check-point. The views can be originaUy acquired slices, any other 
views as described hereinafter in the description of the enhanced presentation 
module 235, or any combination of the above. The user can also associate 
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recommendations for preferred perspectives of the medical imaging device being 
xised during the operation, for better view of the relevant area. The check-points 
and the views and recommendations associated with them are stored in the 
additional images and information storage 30 of Fig. l.The check-points and 
associated views are used during the operation as will be explained in the 
description of the check-point identification and designated views presentation 
module 256 of Fig. 3. 

The non-flexible and curved areas markmg module. 234 enables the 
user to mark parts of blood vessels as non-flexible or curved. Due to the 
volumetric information of the CT data, it is possible to mark a relevant area in an 
image, that is identify location of desired area in CT volume. The marking can 
take place on an original slice or volume, or on the visually presented product of 
processing. The marking is performed, for example, by designating two points 
along a blood vessel so that the part of the blood vessel between these two 
locations is marked as non-flexible. In another embodiment the user freely draws 
the curved line along which the blood vessel is curving. This option is particularly 
useful in the highly curved areas of the blood vessels, or in areas where blood 
vessels branch. As with the check-point definition module, the user can associate 
any desired views with tiie marked areas. The marked areas, their types and the 
associated views are stored in the additional images and information storage 30 of 
Fig. 1. 

The enhanced presentation module 235 complements the standard 
presentation tools module 220. This module presents all the additional 
infomiation deduced by the system and kidicated by the user using the automatic 
modules 22 and the mixed modules 23, oyer the views mentioned above in the 
standard 3-D presentation tools module 220. One type of information included is 
the marking of the different types of plaque layers as deduced by the system in 
the plaque identification and classification module 222 and possibly corrected by 
the user in the plaque correction module 232. Such layers are typically indicated 
by using a designated color for each type of sediment, selected m the parameter 



-16- 



wo 2006/033113 



PCT/IL2005/001024 



setup module 230. Other data include the designated check-points, and the non- 
flexible and curved areas of the blood vessels. Yet more data includes the 
numerical values obtained by the plaque width determination 231, including the 
width of the various plaque layers, the diameter of the blood vessel at the 
specified location, and the percentage of stenosis in that location. The previous 
description relates to the modules and tools available during the pre-operation 
mode. Following are the methods and modules used dvuing the operation, in order 
to fuse and present information gathered prior to and during the operation. 

In a typical non-limiting environment in which the disclosed invention 
is used, a patient is scanned by an MSCT imaging device, and the products of the 
scan are analyzed by a physician or a skilled technician. The results of the 
analysis can include a decision that the patient does need to undergo a 
catheterization, and the products of the pre-operation modules as described in Fig. 
2. 

The previous description relates to the modules and tools available 
during the pre-operation mode. Following are the methods and modules used 
during the operation, in order to fiise and present information gathered prior to 
and during the operation. None of the pre-operation preparations is mandatory. 
All required operations can be performed inmiediately before or during the 
operation. Once the catheterization is in progress, the operating physician takes 
angiograms of the patient The angiograms are taken at different locations, 
perspectives and magnifications according to the physician's needs at any given 
moment during the operation. TTie angiograms locations and angles can also be 
determined prior to the operation by a planning system to get best view of the 
problem. The angiograms imdergo processing yielding 3-dimensional 
reconstructions. The disclosed invention uses images and products of images 
acquired prior to the operation, during the operation. These images and products 
are fused with images and products acquired during the operation. 
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Referring now to Fig 3 that shows the method used in the proposed 
invention for fusing the images and products acquired prior to the operation with 
the images and products acquired during the operation. 

The method comprises the following steps: 
5 In step 239 registration is carried out, meaning establishing 

transformation between objects detected in CT volimie and in angiograms. 

In step 240, global registration is performed, in which the best set of 
parameters defining projection of CT volume into angiographic image plane is 
recovered. The global registration can be carried out in a number of ways. The 

10 first way is the use of calibration devices or fiducials. Fiducials are screws or other 
small objects made of material visible and easily detectible both in the MSCT 
volume and in the angiograms, such as Titanium. The fiducials are attached to the 
patient's body and do not change location between Ihe CT imaging and the 
catheterization procedure, therefore their location in the CT and on the angiogram 

15 disclose the transformation between the two coordinate firames. Another way of 
performing the global registration is by using the parameters supplied by imaging 
system. Yet another option for the global registration involves the usage of 
iterative process of imaging parameter recovering utilizing automatic detection of 
corresponding points in 3-dimensional volume and 2-dimensional projection. One 

20 variant of this process comprises the steps of preparing a synthetic image based on 
projection of CT volume or information extracted fi"om CT volume v^dth 
approximately known imaging parameters; matching of synthetic unage with real 
angiogram using, for example, correlation technique; refinement of imaging 
parameters according to the found local displacements between the two images; 

25 and repeat the steps imtil the process converges to the best imaging parameters. 
Combinations of the abovementioned methods for the global registration can be 
applied as well. 

The global registration process yields for every voxel of CT volume, a 
unique location in the angiographic image. In the opposite direction, generally 
30 every pixel in an angiogram can be mapped into straight line in volume. 
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However, if the pixel belongs to a detected feature in angiogram and is associated 
with certain voxel of the corresponding feature detected in CT volume, the 
correspondence for such pixel is then established. Therefore, matching of 
corresponding features in 3 dimensions and 2 dimensions is an essential part of 
5 establishing a bilateral correspondence. Matching the features is possible due to 
the hierarchy structure and distinctive geometry of blood vessels, i.e. their shapes 
and intersections. If the blood vessels network was denser, such matching might 
not have been possible. Alternatively, if two corresponding pixels are identified in 
two different angiograms then a 3-dimensional location of a corresponding voxel 

10 can be also established. 

In step 241 a local registration is performed which includes removal of 
residual discrepancy between corresponding features detected in CT and 
angiograms. Specifically, the tree of 3-dimensoinal centerlines of blood vessels 
extracted from the CT data is matched with the two-dimensional tree extracted 

15 from two-dimensional angiograms, including branch-to-branch matching on high 
level and point-to-point matching within each matched branches on low level. 
Based on the matching of the trees, the global transfomiation can be augmented 
with continuously changing local correction function. This correction allows the 
establishment of an exact transformation not only for the local features 

20 themselves, but also for neighboring areas. 

In step 242, the in:iages and detailed information acquired prior to the 
operation are fused with the most updated visual information as acquired by the 
angiograms during the operation. The fiision process uses the transformation 
foimd in step 239. The data fusion process starts from a three dimensional image 

25 created from the CT data. The centerlines of the blood vessels are derived from 
the CT data. A fused model combine regions with different resolutions. The 
lumen around the vessel centerlines is presented with its stmcture derived from 
the CT and the high-resolution details originating from angiograms, whereas 
surrounding areas are represented with lower resolution information as acquired 

30 by the CT. Data fusion also takes place when presenting a cross-section of an 



wo 2006/033113 PCT/IL2005/001024 

artery. The approximate shape of the cross-section of the artery is known from 
the CT images, and so are the depositions of plaque. However, since, the 
resolution of the CT is inferior to that of the angio, the vessels boundaries 
information and numeric data, such as the area of the cross section are fused with 
the image and enhance it. The lumen area at any location along the vessel (i.e., 
the area of the cross-section of the blood vessel) is taken from the angio 
information. When the CT cross-section of the blood vessel at the same location 
is zoomed in, the transitions between sediments areas around the limien and the 
lumen itself are fine-tuned to fit the lumen area as determined by the angio. An 
important addition of the angio to the unage fusion is the detection of small 
vessels that are not seen in the CT. The 3-dimensional coordinates of these 
vessels are determined by the 3-dimensional angio system, and thus they are 
fused with the 3- dimensional CT image. 

Referring now to Fig. 4 that shows the options available to a user and 
method of the present invention during the operation. The activities associated 
with tbiese options are performed by the during-operation work station 50 of Fig. 
1, during a medical operation, typically a catheterization. 

When the blooming effect correction option 254 is used, the system 
con-ects the errors caused by the blooming effect, due to which some calcified 
areas look larger in CT images than they should. The error is correctable since the 
angiograms do indicate the correct size of the lumen in areas in which the 
blooming effect in the CT data concealed the lumen. 

The check-point identification and designated views presentation 
option 255, supports using the check-points defined with the pre-operation check- 
point definition and view preparation module 233 of Fig. 2. Whenever the 
coordinates of a check-point are included in an image taken by the angiogram, the 
system automatically uidicates the presence of a check-point and presents the 
views associated with a specific check-point at the pre-operation stage. The 
presence of a check-point in the current angiogram is determined by checking if 
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the coordinates of ttio check-point as projected onto the angio plane are within the 
boundaries of the angio image. 

The enhanced presentation option 256 presents all the images and 
views described in the pre-operation enhanced presentation module 235 of Fig, 2. 
5 In addition, up-to-date angio data acquired diuing the operation is fused with the 
pre-operation images and views to create high-resolution up-to-date three- 
dimensional images. In the following sections, describiag the advanced 
presentation methods, it should be noted that when referring to an image, it is 
either an original image acquired by a device, or a product of processing such 
10 images. For CT images, such products include three-dimensional views of 
vascular trees, surfaces, and the like, plaque indications, check-point indications, 
measurements and the like. For angio images, the products include measurements, 
three-dimensional images of vascular trees acquired from multiple angiograms 
and the like. 

15 In accordance with the preferred embodiment of the present invention, 

three dimensional fused images are presented, in which the "skeleton" or the 
geometry of the blood vessels tree, is taken from the CT images, and the exact 
measurements and high resolution presentation is derived from the angio. Another 
contribution of the CT images to the ftision is the identification of plaque 

20 sediments. The fusion is performed as explained above in step 241 of Fig. 3. 
Another ftision option involves presenting angio images view of 3 -dimensional 
reconstmctions witb. plaque indications derived from the pre-operation stage. The 
indicated plaque layers can incorporate the correction of the blooming effect 
present in the pre-operation stage, by the higher-resolution angiograms. Another 

25 example for ftised elements is the marking of non-flexible or curved areas of the 
vessels as defined in the non-flexible and curved areas marking module 234 of 
Fig. 2, on the angiograms. Yet another example is presenting the plaque layers 
dimensions, the blood vessel's diameter and the stenosis percentage, as enhanced 
during the operation.. 
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In accordance with another preferred embodiment of the present 
invention, images of both devices are viewed side by side. The images can depict 
the same area of the body, different views of the same body area, partly 
overlapping body areas or totally non-overlapping body areas. In another 
preferred embodiment, an image taken by one device depicting a certain area is 
bordered on one or more sides by one or more images of the other device, 
depicting areas which are neighboring the area depicted by the image of the first 
device. The effect of this type of presentation is a continuous view of an area, 
where certain sub-parts of the area wore scanned by one device and the other sub- 
parts were scanned by a second device. In yet another preferred embodiment, 
images taken by a first device prior to the operation and images taken by a second 
device during the operation are presented one on top of the other where the top 
image is at least partially transparent. In another embodiment an image acquired 
by one device, and a larger image acquired by another device are presented where 
the larger image is surrounding the smaller image. The two images can depict the 
same area of the body, neighboring areas or different areas. 

The above shown examples serve merely to provide a clear 
understanding of the invention and not to limit the scope of the present invention 
or the claims appended thereto. Persons skilled in the art will appreciate that 
different or additional modules and methods can be used in association with the 
present invention so as to meet the invention's goals. In particular, different 
methods of fusing and different fused elements can be used. 

The proposed apparatus and methods are innovative in presenting 
during an operation, images and products acquired prior to the operation and 
fusing them with unages and data taken during the operation. The apparatus also 
takes advantage of the developing technology of MSCT devices, which enables 
identification and classification of sediments in blood vessels in general and the 
coronary arteries in particular, and assessment of the percentage and shape of 
stenosis in these blood vessels. This facilitates better assessment of the patient's 
status and aids in the planning and during the execution of a catheterization. 
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The proposed apparatus also facilitates the construction of three- 
dimensional angio images without the interactioix of a human operator. This is 
performed by automatic registration of each angiogram to a two-dimensional 
projection of the CT data, and identifying olDjects appearing both in the 
5 angiogram and in the CT, This, in tum provides th.e matching between the two or 
more angiograms and enables three-dimensional reconstruction from these 
images. 

Persons skilled in the art will appreciate that the present invention can 
also be used with other modalities, such as MRI, once its resolution and scanning 

10 rate enable the identification and classification of plaque. Plaque is identified by 
MR parameters hke Ti, T2. diffusion coefficient, and other MRI tissue 
characteristics. It is also possible to use more tham one set of images, possibly of 
different modalities, prior to the operation, and take the advantages of each of 
them in order to accurately assess the status of the coronaries. For example, one 

15 possible combination is a black blood MRI identifying the plaque with bright 
blood MRI identifying the lumen. Registration of black MRI vs bright MRI is 
done by using the imager common coordinate system. When fusing MR with CT 
images, the registration method of MR and CT is well known in the literature. 

It will be appreciated by persons skilled in the art that the present 

20 invention is not limited to what has been particularly shown and described 
hereinabove. Rather the scope of the present invention is defined only by the 
claims which follow. 
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CLAIMS 

I/we claim: 

1. An apparatus for displaying an at least one first image, said first image is a 
product of processing an at least one second image taken by a first medical 
imaging device prior to a medical operation, tlie first image comprising 
information about areas having sediments, wherein during the operation, 
said first image is presented to a user of the system. 

2. The apparatus of claim 1 where the at least one first image is fused with an 
at least one third image said third image is a pToduct of processing an at 
least one fourth image taken by a second medical imaging device during the 
operation. 

3 . The apparatus of claim 2 further comprising a computer program for: 

registration of images of the first and. second medical imaging 
devices; and 

fusing information contained in and associated with products of 
processing images of the first and second medical imaging devices; and 

presenting an at least one combination image, the combination 
image is selected firom the group consisting of: the at least one first image, 
the at least one third image, a combination of tlae at least one first and the 
at least one third images containing information obtained fi-om the at least 
one fu-st or the at least one second medical imaging devices, an image 
containing information obtained fi-om the first and second medical imaging 
devices. 

4. The apparatus of claim 2 further comprising a correction module to correct 
an at least one imaging error in the at least one second image, using the at 
least one fomlh image acquired during the operation. 

5. The apparatus of claim 4 where the imaging enror is characterized by an at 
least one calcified area of an at least one blood vessel depicted outsized on 
an at least one image, said at least one unage is a product of processing an 
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at least one image taken by an at least one medical imaging device prior to 
an operation, 

6. The apparatus of claim 2 where the at least one first image and the at least 
one third image, are presented on the same location on the visual display, 
where the at least one first and the at least one third images are at least 
partially transparent. 

7. The apparatus of claim 2 where an at least one part of the at least one first 
image, and an at least one part of the at least one third image are presented 
adjacent to each other. 

8. The apparatus of claim 2 where sediments found by processing the at least 
one first image are distinctively marked on the at least one combination 
image. 

9. The apparatus of claim 1 where sediments found by processing the at least 
one first image are distinctively marked on the at least one first image. 

10. The apparatus of claim 1 further comprising a marking module for marking 
an at least one non-flexible part of an at least one blood vessel, on the at 
least one first image. 

1 1 . The apparatus of claim 2 further comprising a marking module for marking 
an at least one non-flexible part of an at least one blood vessel, on the at 
least one combination image. 

12. The apparatus of claim 1 further comprising a module for marking at least 
one curved part of an at least one blood vessel, on the at least one first 
image. 

13. The apparatus of claim 2 further comprising a module for marking at least 
one curved part of an at least one blood vessel, on the at least one 
combination image. 

14. The apparatus of claun 1 further comprising a modiile for marking 
mdications prepared prior to the operation, on thie at least one first image. 
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15. The apparatus of claim 2 further comprising a module for marking 
indications prepared prior to the operation, on the at least one combination 
image. 

16. The apparatus of claim 2 further comprising a module for indicating during 
an operation, perspectives determined prior to the opera^tion, for a medical 
imaging device, said perspectives to be used while taking images during the 
operation. 

17. The apparatus of claim 2 further comprising a module for: 

identifying at least one point in an at least orte image presented 
during an operation with an at least one check-point indicated prior to the 
operation; and 

presenting at least one image, associated prior to the operation with 
the at least one check-point; 

18. The apparatus of claim 1 further comprising a module for the setup of 
system and user preferences . 

19. The apparatus of claim 1 where the blood vessel is a coronary artery. 

20. The apparatus of claim 1 where the sediments are any one of the following: 
lipid-rich plaque, intermediate plaque, calcified plaque, thrombi, cells or 
products of cells. 

21. The apparatus of claim 1 where the first medical imaging device is a multi 
slice computerized tomography device. 

22. The apparatus of claim 1 where the first medical imaging device is a 
magnetic resonance imaging device. 

23. An apparatus for detecting at least one part of at least one blood vessel with 
sediments layers, firom an at least one first image acquired by an at least one 
imaging device prior to an operation, the apparatus comprises: 

an identification module for identifying the at least one part of the at 
least one blood vessel, and within said part the sediments layers located 
therein; and 



-26- 



wo 2006/0331 13 PCT/IL2005/001024 

a marking module for indicating the at least one part of the at least one 
blood vessel and sediment associated therewith on an at least one second 
image created by processing images taken by a medical imaging device. 

24. The apparatus of claim 23 where the identification module is receiving: 

intensity values for at least one pixel of at least one unage acquired 
by a medical imaging device; and 

at least one range of int^isity values for at least one type of 
sediment. 

25. The apparatus of claim 23 further comprising a module for constructing at 
least one visual representation of the lumen of the at least one blood vessel. 

26. The apparatus of claim 23 further comprising a module for constructing an 
at least one visual representation of an at least one part of the wall of the at 
least one blood vessel. 

27. The apparatus of claim 23 wherein parts of the at least one blood vessel and 
the sediment submerged therein are indicated using color-coding. 

28. The apparatus of claim 23 fiirther comprising a width determination module 
for determining: 

the width of the sediments layers at a location along the at least one 
blood vessel; and 

the diameter of the at least one blood vessel at a locatioxi along the 

blood vessel; and 

the percentage of stenosis of the at least one blood vessel at a 
location along the blood vessel. 

29. The apparatus of claim 28 where the widths of the sediment layers, the 
diameter of the blood vessel and the percentage of stenosis are indicated on 
the at least one second image. 

30. The apparatus of claim 23 further comprising a module for indicating, in 
response to a user action, at least one part of an at least one blood vessel as 
non-flexible. 
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31. The apparatus of claim 23 further enabling comprising a module for 
indicating, in response to a user action, at least one part of an at least one 
blood vessel as cxirved. 

32. The apparatus of claim 23 further comprising a check-point definitiori. 
module for indicating, in response to a user action, a position within the 
body of a patient as a check-point and associate said check-point with an at 
least one second image, or an at least one set of perspectives for the medical 
imaging device employed during the operation. 

33. The apparatus of claim 23 where the at least one second image depicts a 
three-dimensional view of the at least one part of the at least one blood 
vessel. 

34. The apparatus of claun 23 where the at least one second image depicts an a.t 
least one surface within the human body and an at least one blood vessel on 
said surface. 

35. The apparatus of claim 23 where the at least one second image depicts an a.t 
least one internal three-dimensional view of the at least one blood vessel. 

36. The apparatus of claim 23 where the at least one second image depicts a 
cross-section of tiie at least one blood vessel at a location along the bloo<i 
vessel, said cross-section comprising one or more of the following: the 
blood vessel's wall, the lumen of the blood vessel, sediments submerged oxi 
the blood vessel's wall. 

37. The apparatus of claun 23 further comprising a module for manuall:y 
coirecting the indications for sediments on unages acquired prior to am 
operation and the products of said images. 

38. The apparatus of claim 37 where the correction includes changing the size 
or the sediment type of the indication, addmg, or deleting indications. 

39. The apparatus of claim 23 where the blood vessel is a coronary artery. 

40. The apparatus of claim 23 where the sediments are any one of ttie 
following: Upid-rich plaque, intermediate plaque, calcified plaque, throml>i, 
cells or products of cells. 
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41. The apparatus of claim 23 where the medical imaging device is a multi slice 
computerized tomography device. 

42. The apparatus of claun 23 where the medical imagmg device is a magnetic 
resonance imaging device. 

43. A method for displaying an at least one first unage, said first image is a 
product of processing at least one second image taken by a first medical 
imaging device prior to a medical operation, the image comprismg 
information about areas with sediments, dvuing the operation. 

44. The method of claim 0 where the at least one fiirst image is fused with an at 
least one third image which is a product of processmg an at least one fourth 
image taken by a second medical unaging device during the operation, the 
method comprises the following steps: 

registering the coordinate systems of the first and the third images; 

and 

fusing information contained in and associated with the at least one 
first image and the at least one third image; and 

presentmg an at least one combination image, the combination 
unage is selected fi-om the group consisting of: the at least one first image, 
the at least one thdrd image, a combination of the at least one first and the 
at least one third unages containing information obtained firom the at least 
one first or the at least one second medical unaging devices, an knage 
containing mfoimation obtained fi-om the first and second medical imaging 
devices. 

45. The method of claim 44 where the registration of the coordinate systems 
comprises the following steps: 

global registration of the first image and the third image; 
removal of local residual discrepancies by matching 
corresponding features detected in the first and the thkd unages. 
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46. The method of claun 45 where the global registration is based on 
comparing the coordinates of at least one fiducial as seen in the first and the 
third images. 

47. The method of claim 45 where the global registration is based on matching 
an at least one third image to an at least one projection of a three 
dimensional data obtained firom the at least one first image prior to the 
operation. 

48. The method of claim 44 further comprising a step of correcting an at least 
one imaging error in the at least first koage, using the at least one third 
image 

49. The method of claim 48 where the at least one imaging error is 
characterized by an at least one calcified area of an at least one blood vessel 
depicted outsized on an at least one product of processing of at least one 
image taken by a medical imaging device prior to an operation. 

50. The method of claim 44 where the at least one first image and the at. least 
one third image, are presented on the same location on the visual display, 
where the first and the third images are at least partially transparent. 

51. The method of claim 44 where at least one part of the at least one first 
image, and at least one part of the at least one third image are presented 
adjacent to each other. 

52. The method of claim 44 where sediments found by processing the at least 
one first image are marked on the at least one third image. 

53. The method of claim 44 further comprising a step of marking at least one 
non-flexible part of an at least one blood vessel, on the at least one first 
image. 

54. The method of claim 44 further comprising a step of marking at least one 
non-flexible part of an at least one blood vessel, on the at least one third 
image. 

55. The method of claim 44 further comprising a step of marking at least one 
curved portion of an at least one blood vessel, on the at least one first image. 
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56. The method of claim 44 further comprising a step of marking at least one 
curved portion of ah at least one blood vessel, on the at least one third 
image. 

57. The method of claim 44 further comprising the steps of: 

identifying at least one point in an at least one image acquired 
during the operation with an at least one check-point indicated prior to the 
operation; and 

presenting at least one image associated prior to the operation with 
the at least one check-point. 

58. The method of claim 0 where the blood vessel is a coronary artery. 

59. The method of 0 where the sediments are lipid-rich plaque, intermediate 
plaque, calcified plaque, thrombi, cells or products of cells. 

60. The method of 0 where the medical imagmg device is a multi-sHce 
computerized tomography device. 

61. The method of 0 where the medical imaging device is a magnetic resonance 
imaging device. 

62. A method for automatically detecting an at least one area of an at least one 
blood vessel having sediment submerged thareto, said sediment is of one or 
more types, using at least one first unage acquired by an at least one 
medical imaging device, the method comprising the steps of : 

identifying the at least one area of the at least one blood 
vessel with sediments layers; and 

indicating the at least one area on an at least one second 
image, said second image is depicting a product of processing the at 
least one first image. 

63. The method of claim 62 where the identification step comprises the step of 
comparing the intensity value of an at least one pixel of an at least one 
image acquired by a medical imaging device to at least one range of 
intensity values for an at least one type of sediment. 
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64. The method of claim 62 further comprising the step of constructing an at 
least one visual representation of the lumen of the at least one blood vessel; 

65. The method of claim 62 further comprising the step of constructing an at 
least one visual representation of an at least one part of an at least one blood 
vessel. 

66. The method of claim 62 where sediments submerged in the at least one 
blood vessel are indicated using color-codings 

67. The method of claim 62 further comprising the steps of determining any 
one of the following: 

the width of the sediment layers at a position along the at least one 

blood vessel; and 

the diameter of the blood vessel at a position along the at least one 

blood vessel; and 

the percentage of stenosis of the at least one blood vessel at a 

position along the blood vessel. 

68. The method of claim 67 further comprising the step of indicating on the 
second unage the at least one width of the sediments layers, the diameter of 
the at least one blood vessel and the percentage of stenosis. 

69. The method of claim 62 fiirther comprising a step of marking on the second 
image, in response to a user's actions, at least one part of an at least one 
blood vessel as non-flexible. 

70. The method of claim 62 further comprising a step of marking on the second 
image, in response to a user's actions, at least one part of an at least one 
blood vessel as bemg curved. 

71. The method of claim 62 further comprising a step of indicating on the 
second image, in response to a user's actions, a point within the body of a 
patient as a check-point and associate said check-point with an at least one 
image, said unage is a product of processing unages taken by a medical 
imaging device prior to an operation, or an at least one set of perspectives 
for the medical imaging device employed during the operation. 
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72. The method of claim 62 wherein the at least one second image depicts an at 
least one three-dimensional view of the at least one blood vessel, 

73. The method of claim 62 where the at least one second image depicts an at 
least one three-dimensional surface within the human body. 

74. The method of claim 62 where the at least one second image depicts an 
internal three-dimensional view of a coronary artery. 

75. The method of claim 62 where the at least one second image depicts a 
cross-section of the at least one blood vessel, at a location along the at least 
one blood vessel, said cross-section comprising any one of the following: 
the blood vessel wall, the lumen of the blood vessel, sediment. 

76. The method of claim 62 further comprising the step of providing a user 
with the option to manually correct the indications for sediments on images 
acquired prior to an operation and on the products of processmg said 
images. 

77. The method of claim 62 where the correction includes any one of the 
following: changing the size or the sediment type of an indication, adding, 
or deleting indications. 

78. The method of claim 62 where the blood vessel is a coronary artery. 

79. The method of claim 62 where the sediments are lipid-rich plaque, 
intermediate plaque, calcified plaque, thrombi, cells or products of cells. 

80. The method of claim 62 where the medical imaging device is a muW slice 
computerized tomography device. 

81. The method of claim 62 where the medical imaging device is a magnetic 
resonance imaging device. 

82. A method for automatic reconstruction of a three-dimensional object from 
two angiograms using information collected from a modality, the method 
comprises the following steps: 

taking a first and a second angiograms of the required area 
from different perspectives; and 
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for the first and the second angiogram, obtaining a first and a 
second projected images by projecting data collected firom the 
modality on the same plane as the first and the second angiogram; 
and 

5 registration of the first and the second angiogram with the 

corresponding projected images by objects appearing in the first or 
the second angiogram and in the first or second projected image; 
and 

mutual co-registration of the first and the second 

10 angiograms; and 

detecting objects appearing in the fibrst or tiie second 
angiogram and match with the corresponding objects in the first or 
second projected image; and 

deriviag the three dimensional coordinates of the objects 
15 appearing in the fij^it and the second angiograms; and 

constmcting a three dimensional image of the required area 
fi-om the first and the second angiogram. 
83. The method of claim 82 wherein the modaUty is a computerized 
tomography imaging device. 
20 84. The method of claim 82 wherein the modality is a magnetic resonance 

imaging device. 
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